Most computers today, especially the ubiquitous personal computers, can be configured with any number of devices from a variety of vendors. For example, the typical personal computer can be configured from a variety of graphic display systems, scanners, input devices such as a mouse or digitizing pad, network interface cards, and Web cameras, to name just a few. Indeed, the possibilities often seem endless.
Competition among the various vendors is often quite fierce. In order to attract buyers, vendors frequently promote their products according to the speed with which they can complete a certain task or tasks on a given computer system. For example, with respect to graphic display systems, third-party vendors often promote their graphic display systems according to speed-related benchmarks achieved by their product when run against a benchmark suite. Industry-related magazines and publications often review, promote, and/or endorse products based on these benchmark results. Correspondingly, many users purchase third-party devices based on the various benchmarks published or associated with a device. Almost universally, benchmarks are speed-related evaluations.
As those familiar with the computer industry will recognize, third-party vendors often analyze the benchmark suite, examining and analyzing the methodologies by which the benchmarks are derived. Such analysis provides the vendors with invaluable insight as to what exactly is sought by reviewers (and presumably customers), and also provides insight as to how they may improve their products to achieve better benchmark scores. Unfortunately, this information often leads a vendor to optimize their product for benchmark purposes, frequently turning to complex processing, short-cuts, and raising (even exceeding) operating limits. Whether or not a vendor intentionally does so, these “optimizations” tend to sacrifice operating stability for benchmark performance, i.e., speed.
One area that is often sacrificed to optimization for performance sake is the interface between a computer's operating system and the hardware device, hereafter referred to as the “device driver.” Indeed, the device driver is a logical choice for optimization, as its modification represents the least expensive course, and one that could yield substantial results, for a vendor. Unfortunately, device driver failures account for a substantial percentage of computer system crashes. According to some statistics, at least one out of every four system crashes is attributable to a device driver malfunction.
While market forces typically drive vendors to optimize their products for improved speed, there is an unfortunate lack of a complimentary force that encourages vendors to produce stable products, i.e., those that will not cause system crashes. In other words, there is currently no system available that benchmarks or measures the stability of a product in the same way that benchmarks are available for speed/performance. It should be noted that many operating system providers provide a compatibility certification for third party products, including devices. In other words, if a vendor meets the established baseline of compatibility with an operating system, the operating system provider will permit the vendor to advertise that its product has been certified as compatible with the operating system. Unfortunately, such certification simply indicates that a product has met some minimum level of compatibility, and does not provide a consumer with any information by which a comparison may be made between products. In fact, there is a general need in the computer industry for a stability benchmark rating of products. The present invention addresses these needs and other issues found in the prior art.